Functional Analysis

Gerald Teschl

Gerald Teschl Institut für Mathematik Nordbergstraße 15 Universität Wien 1090 Wien, Austria

E-mail address: Gerald.Teschl@univie.ac.at
URL: http://www.mat.univie.ac.at/~gerald/

1991 Mathematics subject classification. 81-01, 81Qxx

Abstract. This manuscript provides a brief introduction to Functional Analysis.

Warning: This is an incomplete DRAFT!

Keywords and phrases. Functional Analysis, Banach space, Hilbert space.

Typeset by AMS-L^{A}TEX and Makeindex. Version: October 18, 2004 Copyright \begin{matrix}c \\ ◯ \end{matrix} 2004 by Gerald Teschl

Contents

Preface  v
Chapter 0. Introduction  1
§0.1. Linear partial differential equations  1
Chapter 1. A first look at Banach and Hilbert spaces  5
§1.1. The Banach space of continuous functions  5
§1.2. The geometry of Hilbert spaces  9
§1.3. Completeness  13
§1.4. Bounded operators  14
Chapter 2. Hilbert spaces  17
§2.1. Orthonormal bases  17
§2.2. The projection theorem and the Riesz lemma  21
§2.3. Orthogonal sums and tensor products  23
§2.4. Compact operators  24
§2.5. The spectral theorem for compact symmetric operators  26
§2.6. Applications to Sturm-Liouville operators  28
Chapter 3. Banach spaces  31
Bibliography  33
Glossary of notations  35
Index  36
Index  37

Preface

The present manuscript was written for my course Functional Analysis given at the University of Vienna in Winter 2004.

It is available from

       http://www.mat.univie.ac.at/~gerald/ftp/book-fa/

Acknowledgments

I’d like to thank ....

Gerald Teschl

Vienna, Austria October, 2004

Chapter 0

Introduction

Functional analysis is an important tool in the investigation of all kind of problems in pure mathematics, physics, biology, economics, etc.. In fact, it is hard to find a branch in science where functional analysis is not used.

The main objects are (infinite dimensional) linear spaces with different concepts of convergence. The classical theory focuses on linear operators (i.e., functions) between these spaces but nonlinear operators are of course equally important. However, since one of the most important tools in investigating nonlinear mappings is linearization (differentiation), linear functional analysis will be our first topic in any case.

0.1. Linear partial differential equations

Rather than overwhelming you with a vast number of classical examples I want to focus on one: linear partial differential equations. We will use this example as a guide throughout this first chapter and will develop all necessary method for a successful treatment of our particular problem.

In his investigation of heat conduction Fourier was lead to the (one dimensional) heat or diffusion equation

\frac{∂}{∂t}u(t, x) = \frac{∂^{2}}{∂x^{2}} u(t, x),  (0.1)

Here u(t, x) is the temperature distribution at time t at the point x. It is usually assumed, that the temperature at x = 0 and x = 1 is fixed, say u(t, 0) = a and u(t, 1) = b. By considering u(t, x) → u(t, x)−a−(b−a)x it is clearly no restriction to assume a = b = 0. Moreover, the initial temperature distribution u(0, x) = u_{0}(x) is assumed to be know as well.

Since finding the solution seems at first sight not possible, we could try to find at least some some solutions of (0.1) first. We could for example make an ansatz for u(t, x) as a product of two functions, each of which depends on only one variable, that is,

u(t, x) = w(t)y(x).  (0.2)

This ansatz is called separation of variables. Plugging everything into the heat equation and bringing all t, x dependent terms to the left, right side, respectively, we obtain

\frac{ẇ(t)}{w(t)} = \frac{y′′(x)}{y(x)}.  (0.3)

Here the dot refers to differentiation with respect to t and the prime to differentiation with respect to x.

Now if this equation should hold for all t and x, the quotients must be equal to a constant −λ. That is, we are lead to the equations

−ẇ(t) = λw(t)  (0.4)

and

−y′′(x) = λy(x),  y(0) = y(1) = 0  (0.5)

which can easily be solved. The first one gives

w(t) = c_{1}e^{−λt}  (0.6)

and the second one

y(x) = c_{2} cos(\sqrt{λ}x) + c_{3} sin(\sqrt{λ}x).  (0.7)

However, y(x) must also satisfy the boundary conditions y(0) = y(1) = 0. The first one y(0) = 0 is satisfied if c_{2} = 0 and the second one yields (c_{3} can be absorbed by w(t))

sin(\sqrt{λ}) = 0,  (0.8)

which holds if λ = (πn)^{2}, n ∈ ℕ. In summary, we obtain the solutions

u_{n}(t, x) = c_{n}e^{−(πn)^{2}t} sin(nπx),  n ∈ ℕ.  (0.9)

So we have found a large number of solutions, but we still have not dealt with our initial condition u(0, x) = u_{0}(x). This can be done using the superposition principle which holds since our equation is linear. In fact, choosing

u(t, x) = ∑_{n=1}^{∞} c_{n}e^{−(πn)^{2}t} sin(nπx),  (0.10)

where the coefficients c_{n} decay sufficiently fast, we obtain further solutions of our equation. Moreover, these solutions satisfy

u(0, x) = ∑_{n=1}^{∞} c_{n} sin(nπx)  (0.11)

and expanding the initial conditions into Fourier series

u_{0}(x) = ∑_{n=1}^{∞} u_{0,n} sin(nπx),  (0.12)

we see that the solution of our original problem is given by (0.10) if we choose c_{n} = u_{0,n}.

Of course for this last statement to hold we need to ensure that the series in (0.10) converges and that we can interchange summation and differentiation. You are asked to do so in Problem 0.1.

In fact many equations in physics can be solved in a similar way:

• Reaction-Diffusion equation:

\frac{∂}{∂t}u(t, x) − \frac{∂^{2}}{∂x^{2}} u(t, x) + q(x)u(t, x) = 0,

u(0, x) = u_{0}(x),

u(t, 0) = u(t, 1) = 0.  (0.13)

Here u(t, x) could be the density of some gas in a pipe and q(x) > 0 describes that a certain amount per time is removed (e.g., by a chemical reaction).

• Wave equation:

\frac{∂^{2}}{∂t^{2}} u(t, x) − \frac{∂^{2}}{∂x^{2}} u(t, x) = 0,

u(0, x) = u_{0}(x), \frac{∂u}{∂t}(0, x) = v_{0}(x)

u(t, 0) = u(t, 1) = 0.  (0.14)

Here u(t, x) is the displacement of a vibrating string which is fixed at x = 0 and x = 1. Since the equation is of second order in time, both the initial displacement u_{0}(x) and the initial velocity v_{0}(x) of the string need to be known.

• Schrödinger equation:

i\frac{∂}{∂t}u(t, x) = −\frac{∂^{2}}{∂x^{2}} u(t, x) + q(x)u(t, x),

u(0, x) = u_{0}(x),

u(t, 0) = u(t, 1) = 0.  (0.15)

Here |u(t, x)|^{2} is the probability distribution of a particle trapped in a box x ∈ [0, 1] and q(x) is a given external potential which describes the forces acting on the particle.

All these problems (and many others) leads to the investigation of the following problem

Ly(x) = λy(x),  L = −\frac{d^{2}}{dx^{2}} + q(x),  (0.16)

subject to the boundary conditions

y(a) = y(b) = 0.  (0.17)

Such a problem is called Sturm–Liouville boundary value problem. Our example shows that we should prove the following facts about our Sturm–Liouville problems:

(1) The Sturm–Liouville problem has a countable number of eigenvalues E_{n} with corresponding eigenfunctions u_{n}(x), that is, u_{n}(x) satisfies the boundary conditions and Lu_{n}(x) = E_{n}u_{n}(x).

(2) The eigenfunctions u_{n} are complete, that is, any nice function u(x) can be expanded into a generalized Fourier series

u(x) = ∑_{n=1}^{∞} c_{n}u_{n}(x).

This problem is very similar to the eigenvalue problem of a matrix and we are looking for a generalization of the well-known fact that every symmetric matrix has an orthonormal basis of eigenvectors. However, our linear operator L is now acting on some space of functions which is not finite dimensional and it is not at all what even orthogonal should mean for functions. Moreover, since we need to handle infinite series, we need convergence and hence define the distance of two functions as well.

Hence our program looks as follows:

• What is the distance of two functions? This automatically leads us to the problem of convergence and completeness.

• If we additionally require the concept of orthogonality, we are lead to Hilbert spaces which are the proper setting for our eigenvalue problem.

• Finally, the spectral theorem for compact symmetric operators will be the solution of our above problem

Problem 0.1. Find conditions for the initial distribution u_{0}(x) such that (0.10) is indeed a solution (i.e., such that interchanging the order of summation and differentiation is admissible).

A first look at Banach and Hilbert spaces

1.1. The Banach space of continuous functions

So let us start with the set of continuous function C(I) on a compact interval I = [a, b] ⊂ ℝ. Since we want to handle complex models (e.g., the Schrödinger equation) as well, we will always consider complex valued functions!

One way of declaring a distance, well-known from calculus, is the maximum norm:

∥f(x) − g(x)∥_{∞} = max_{x∈I} |f(x) − g(x)|.  (1.1)

It is not hard to see that with this definition C(I) becomes a normed linear space:

A normed linear space X is a vector space X over ℂ (or ℝ) with a real-valued function (the norm) ∥.∥ such that

• ∥f∥ ≥ 0 for all f ∈ X and ∥f∥ = 0 if and only if f = 0,

• ∥λ f∥ = |λ| ∥f∥ for all λ ∈ ℂ and f ∈ X, and

• ∥f + g∥ ≤ ∥f∥ + ∥g∥ for all f, g ∈ X (triangle inequality).

Once we have a norm, we have a distance d(f, g) = ∥f − g∥ and hence we know when a sequence of vectors f_{n} converges to a vector f. We will write f_{n} → f or lim_{n→∞} f_{n} = f, as usual, in this case. Moreover, a mapping F : X → Y between to normed spaces is called continuous if f_{n} → f implies F (f_{n}) → F (f). In fact, it is not hard to see that the norm is continuous (Problem 1.2).

In addition to the concept of convergence we have also the concept of a Cauchy sequence and hence the concept of completeness: A normed space is called complete if every Cauchy sequence has a limit. A complete normed space is called a Banach space. Example. The space ℓ^{1}(ℕ) of all sequences a = (a_{j})_{j=1}^{∞} for which the norm

∥a∥_{1} = ∑_{j=1}^{∞} |a_{j}|  (1.2)

is finite, is a Banach space.

To show this, we need to verify three things: (i) ℓ^{1}(ℕ) is a Vector space, that is closed under addition and scalar multiplication (ii) ∥.∥_{1} satisfies the three requirements for a norm and (iii) ℓ^{1}(ℕ) is complete.

First of all observe

∑_{j=1}^{k} |a_{j} + b_{j}| ≤ ∑_{j=1}^{k} |a_{j}| + ∑_{j=1}^{k} |b_{j}| ≤ ∥a∥_{1} + ∥b∥_{1}  (1.3)

for any finite k. Letting k → ∞ we conclude that ℓ^{1}(ℕ) is closed under addition and that the triangle inequality holds. That ℓ^{1}(ℕ) is closed under scalar multiplication and the two other properties of a norm are straightforward. It remains to show that ℓ^{1}(ℕ) is complete. Let a^{n} = (a_{j}^{n})_{j=1}^{∞} be a Cauchy sequence, that is, for given ε > 0 we can find an N_{ε} such that ∥a^{m} − a^{n}∥_{1} ≤ ε for m, n ≥ N_{ε}. This implies in particular |a_{j}^{m} − a_{j}^{n}| ≤ ε for any fixed j. Thus a_{j}^{n} is a Cauchy sequence for fixed j and by completeness of ℂ has a limit: a_{j}^{n} → a_{j}. Now consider

∑_{j=1}^{k} |a_{j}^{m} − a_{j}^{n}| ≤ ε  (1.4)

and take m → ∞:

∑_{j=1}^{k} |a_{j} − a_{j}^{n}| ≤ ε.  (1.5)

Since this holds for any finite k we even have ∥a−a^{n}∥_{1} ≤ ε. Hence (a−a^{n}) ∈ ℓ^{1}(ℕ) and since a^{n} ∈ ℓ^{1}(ℕ) we finally conclude a = a^{n} + (−a^{n}) ∈ ℓ^{1}(ℕ). ⋄

Example. The space ℓ^{∞}(ℕ) of all bounded sequences a = (a_{j})_{j=1}^{∞} together with the norm

∥a∥_{∞} = sup_{j∈ℕ}|a_{j}|  (1.6)

is a Banach space (Problem 1.3). ⋄

Now what about convergence in this space? A sequence of functions f_{n}(x) converges to f if and only if

lim_{n→∞} ∥f − f_{n}∥ = lim_{n→∞} sup_{x∈I} |f_{n}(x) − f(x)| = 0.  (1.7)

That is, in the language of real analysis, f_{n} converges uniformly to f. Now let us look at the case where f_{n} is only a Cauchy sequence. Then f_{n}(x) is clearly a Cauchy sequence of real numbers for any fixed x ∈ I. In particular, by completeness of ℂ, there is a limit f(x) for each x. Thus we get a limiting function f(x). Moreover, letting m → ∞ in

|f_{m}(x) − f_{n}(x)| ≤ ε  ∀m, n > N_{ε}, x ∈ I  (1.8)

we see

|f(x) − f_{n}(x)| ≤ ε ∀n > N_{ε}, x ∈ I, (1.9)

that is, f_{n}(x) converges uniformly to f(x). However, up to this point we don’t know whether it is in our vector space C(I) or not, that is, whether it is continuous or not. Fortunately, there is a well-known result from real analysis which tells us that the uniform limit of continuous functions is again continuous. Hence f(x) ∈ C(I) and thus every Cauchy sequence in C(I) converges. Or, in other words

Theorem 1.1. C(I) with the maximum norm is a Banach space.

Next we want to know if there is a basis for C(I). In order to have only countable sums, we would even prefer a countable basis. If such a basis exists, that is, if there is a set {u_{n}} ⊂ X of linearly independent vectors such that every element f ∈ X can be written as

f = ∑_{n} c_{n}u_{n},  c_{n} ∈ ℂ,  (1.10)

then the span (the set of all linear combinations) of {u_{n}} is dense in X. A set whose span is dense is called total and if we have a total set, we also have a countable dense set (consider only linear combinations with rational coefficients). A normed linear space containing a countable dense set is called separable. Luckily this is the case for C(I):

Theorem 1.2 (Weierstraß). Let I be a compact interval. Then the set of polynomials is dense in C(I).

Proof. Let f(x) ∈ C(I) be given. By considering f(x) − f(a) + (f(b) − f(a))(x − b) it is no loss to assume that f vanishes at the boundary points. Moreover, without restriction we only consider I = [\frac{−1}{2}, \frac{1}{2}] (why?).

Now the claim follows from the lemma below using

u_{n}(x) = \frac{1}{I_{n}} (1 − x^{2})^{n},  (1.11)

where

I_{n} = ∫_{−1}^{1}(1 − x^{2})^{n}dx = \frac{n!}{\frac{1}{2}(\frac{1}{2} + 1) ⋯ (\frac{1}{2} + n)}
= \sqrt{π} \frac{Γ(1 + n)}{Γ(\frac{3}{2} + n)} = \sqrt{\frac{π}{n}}(1 + O(\frac{1}{n})).  (1.12)

(Remark: The integral is known as Beta function and the asymptotics follow from Stirling’s formula.) □

Lemma 1.3 (Smoothing). Let u_{n}(x) be a sequence of nonnegative continuous functions on [−1, 1] such that

∫_{|x|≤1} u_{n}(x)dx = 1 and ∫_{δ≤|x|≤1} u_{n}(x)dx → 0, δ > 0.  (1.13)

(In other words, u_{n} has mass one and concentrates near x = 0 as n → ∞.)

Then for every f ∈ C[−\frac{1}{2}, \frac{1}{2}] which vanishes at the endpoints, f(−\frac{1}{2}) = f(\frac{1}{2}) = 0, we have that

f_{n}(x) = ∫_{−1/2}^{1/2} u_{n}(x − y)f(y)dy  (1.14)

converges uniformly to f(x).

Proof. Since f is uniformly continuous, for given ε we can find a δ (independent of x) such that |f(x)−f(y)| ≤ ε whenever |x−y| ≤ δ. Moreover, we can choose n such that ∫_{δ≤|y|≤1} u_{n}(y)dy ≤ ε. Now abbreviate M = max f and note

|f(x)−∫_{−1/2}^{1/2} u_{n}(x−y)f(x)dy| = |f(x)| |1−∫_{−1/2}^{1/2} u_{n}(x−y)dy| ≤ Mε. (1.15)

In fact, either the distance of x to one of the boundary points ±\frac{1}{2} is smaller than δ and hence |f(x)| ≤ ε or otherwise the difference between one and the integral is smaller than ε.

Using this we have

|f_{n}(x) − f(x)| ≤ ∫_{−1/2}^{1/2} u_{n}(x − y)|f(y) − f(x)|dy + Mε
≤ ∫_{|y|≤1/2,|x−y|≤δ} u_{n}(x − y)|f(y) − f(x)|dy

+ ∫_{|y|≤1/2,|x−y|≥δ} u_{n}(x − y)|f(y) − f(x)|dy + Mε

= ε + 2Mε + Mε = (1 + 3M)ε,  (1.16)

which proves the claim. □

Note that f_{n} will be as smooth as u_{n}, hence the title smoothing lemma. The same idea is used to approximate noncontinuous functions by smooth ones (of course the convergence will no longer be uniform in this case).

Corollary 1.4. C(I) is separable.

The same is true for ℓ^{1}(ℕ), but not for ℓ^{∞}(ℕ) (Problem 1.4)!

Problem 1.1. Show that |∥f∥ − ∥g∥| ≤ ∥f − g∥.

Problem 1.2. Show that the norm, vector addition, and multiplication by scalars are continuous. That is, if f_{n} → f, g_{n} → g, and λ_{n} → λ then ∥f_{n}∥ → ∥f∥, f_{n} + g_{n} → f + g, and λ_{n}g_{n} → λg.

Problem 1.3. Show that ℓ^{∞}(ℕ) is a Banach space.

Problem 1.4. Show that ℓ^{1}(ℕ) is separable. Show that ℓ^{∞}(ℕ) is not separable (Hint: Consider sequences which take only the value one and zero. How many are there? What is the distance between two such sequences?).

1.2. The geometry of Hilbert spaces

So it looks like C(I) has all the properties we want. However, there is still one thing missing: How should we define orthogonality in C(I)? In Euclidean space, two vectors are called orthogonal if their scalar product vanishes, so we would need a scalar product:

Suppose ℌ is a vector space. A map ⟨., ..⟩ : ℌ × ℌ → ℂ is called skew linear form if it is conjugate linear in the first and linear in the second argument, that is,

\begin{aligned}⟨λ_{1}f_{1} + λ_{2}f_{2}, g⟩ & = & λ_{1}^{∗}⟨f_{1}, g⟩ + λ_{2}^{∗}⟨f_{2}, g⟩ \\ ⟨f, λ_{1}g_{1} + λ_{2}g_{2}⟩ & = & λ_{1}⟨f, g_{1}⟩ + λ_{2}⟨f, g_{2}⟩ \end{aligned} , λ_{1}, λ_{2} ∈ ℂ,  (1.17)

where ‘∗’ denotes complex conjugation. A skew linear form satisfying the requirements

(1) ⟨f, f⟩ > 0 for f ≠ 0.
(2) ⟨f, g⟩ = ⟨g, f⟩^{∗}

is called inner product or scalar product. Associated with every scalar product is a norm

∥f∥ = \sqrt{⟨f, f⟩}.  (1.18)

The pair (ℌ, ⟨., ..⟩) is called inner product space. If ℌ is complete it is called a Hilbert space. Example. Clearly ℂ^{n} with the usual scalar product

⟨a, b⟩ = ∑_{j=1}^{n} a_{j}^{∗}b_{j}  (1.19)

is a (finite dimensional) Hilbert space. ⋄

Example. A somewhat more interesting example is the Hilbert space ℓ^{2}(ℕ), that is, the set of all sequences

{(a_{j})_{j=1}^{∞}| ∑_{j=1}^{∞} |a_{j}|^{2} < ∞}  (1.20)

with scalar product

⟨a, b⟩ = ∑_{j=1}^{∞} a_{j}^{∗}b_{j}.  (1.21)

(Show that this is in fact a separable Hilbert space! Problem 1.5) ⋄

Of course I still owe you a proof for the claim that \sqrt{⟨f, f⟩} is indeed a norm. Only the triangle inequality is nontrivial which will follow from the Cauchy-Schwarz inequality below.

A vector f ∈ ℌ is called normalized or unit vector if ∥f∥ = 1. Two vectors f, g ∈ ℌ are called orthogonal or perpendicular (f ⊥ g) if ⟨f, g⟩ = 0 and parallel if one is a multiple of the other.

For two orthogonal vectors we have the Pythagorean theorem:

∥f + g∥^{2} = ∥f∥^{2} + ∥g∥^{2},  f ⊥ g,  (1.22)

which is one line of computation.

Suppose u is a unit vector, then the projection of f in the direction of u is given by

f_{∥} = ⟨u, f⟩u  (1.23)

and f_{⊥} defined via

f_{⊥} = f − ⟨u, f⟩u  (1.24)

is perpendicular to u since ⟨u, f_{⊥}⟩ = ⟨u, f −⟨u, f⟩u⟩ = ⟨u, f⟩−⟨u, f⟩⟨u, u⟩ = 0.

[figure: f; f_{∥}; f_{⊥}; u]

Taking any other vector parallel to u it is easy to see

∥f − λu∥^{2} = ∥f_{⊥} + (f_{∥} − λu)∥^{2} = ∥f_{⊥}∥^{2} + |⟨u, f⟩ − λ|^{2}  (1.25)

and hence f_{∥} = ⟨u, f⟩u is the unique vector parallel to u which is closest to f.

As a first consequence we obtain the Cauchy-Schwarz-Bunjakowski inequality:

Theorem 1.5 (Cauchy-Schwarz-Bunjakowski). Let ℌ_{0} be an inner product space, then for every f, g ∈ ℌ_{0} we have

|⟨f, g⟩| ≤ ∥f∥∥g∥  (1.26)

with equality if and only if f and g are parallel.

Proof. It suffices to prove the case ∥g∥ = 1. But then the claim follows from ∥f∥^{2} = |⟨g, f⟩|^{2} + ∥f_{⊥}∥^{2}. □

Note that the Cauchy-Schwarz inequality implies that the scalar product is continuous in both variables, that is, if f_{n} → f and g_{n} → g we have ⟨f_{n}, g_{n}⟩ → ⟨f, g⟩.

As another consequence we infer that the map ∥.∥ is indeed a norm.

∥f + g∥^{2} = ∥f∥^{2} + ⟨f, g⟩ + ⟨g, f⟩ + ∥g∥^{2} ≤ (∥f∥ + ∥g∥)^{2}.  (1.27)

But let us return to C(I). Can we find a scalar product which has the maximum norm as associated norm? Unfortunately the answer is no! The reason is that the maximum norm does not satisfy the parallelogram law (Problem 1.7).

Theorem 1.6 (Jordan-von Neumann). A norm is associated with a scalar product if and only if the parallelogram law

∥f + g∥^{2} + ∥f − g∥^{2} = 2∥f∥^{2} + 2∥g∥^{2}  (1.28)

holds.

In this case the scalar product can be recovered from its norm by virtue of the polarization identity

⟨f, g⟩ = \frac{1}{4} (∥f + g∥^{2} − ∥f − g∥^{2} + i∥f − ig∥^{2} − i∥f + ig∥^{2}) .  (1.29)

Proof. If an inner product space is given, verification of the parallelogram law and the polarization identity is straight forward (Problem 1.6).

To show the converse, we define

s(f, g) = \frac{1}{4} (∥f + g∥^{2} − ∥f − g∥^{2} + i∥f − ig∥^{2} − i∥f + ig∥^{2}) .  (1.30)

Then s(f, f) = ∥f∥^{2} and s(f, g) = s(g, f)^{∗} are straightforward to check. Moreover, another straightforward computation using the parallelogram law shows

s(f, g) + s(f, h) = 2s(f, \frac{g + h}{2}).  (1.31)

Now choosing h = 0 (and using s(f, 0) = 0) shows s(f, g) = 2s(f, \frac{g}{2}) and thus s(f, g) + s(f, h) = s(f, g + h). Furthermore, by induction we infer \frac{m}{2^{n}} s(f, g) = s(f, \frac{m}{2^{n}}), that is λs(f, g) = s(f, λg) for every positive rational λ. By continuity (check this!) this holds for all λ > 0 and s(f, −g) = −s(f, g) respectively s(f, ig) = i s(f, g) finishes the proof. □

Note that the parallelogram law and the polarization identity even hold for skew linear forms.

But how do we define a scalar product on C(I)? One possibility is

⟨f, g⟩ = ∫_{a}^{b} f^{∗}(x)g(x)dx.  (1.32)

The corresponding inner product space is denoted by L^{2}(I). Note that we have

∥f∥ ≤ \sqrt{|b − a|}∥f∥_{∞}  (1.33)

and hence the maximum norm is stronger than the L^{2} norm.

Suppose we have two norms ∥.∥_{1} and ∥.∥_{2} on a space X. Then ∥.∥_{2} is said to be stronger than ∥.∥_{1} if there is a constant m > 0 such that

∥f∥_{1} ≤ m∥f∥_{2}.  (1.34)

It is straightforward to check that

Lemma 1.7. If ∥.∥_{2} is stronger than ∥.∥_{1}, then any ∥.∥_{2} Cauchy sequence is also a ∥.∥_{1} Cauchy sequence.

Hence if a function F : X → Y is continuous in (X, ∥.∥_{1}) it is also continuos in (X, ∥.∥_{2}) and if a set is dense in (X, ∥.∥_{2}) it is also dense in (X, ∥.∥_{1}).

In particular, L^{2} is separable. But is it also complete? Unfortunately the answer is no: Example. Take I = [0, 2] and define

f_{n}(x) = \{ \begin{matrix}0, & 0 ≤ x ≤ 1 − \frac{1}{n} \\ 1 + n(x − 1), & 1 − \frac{1}{n} ≤ x ≤ 1 \\ 1, & 1 ≤ x ≤ 2 \end{matrix}  (1.35)

then f_{n}(x) is a Cauchy sequence in L^{2}, but there is no limit in L^{2}! Clearly the limit should be the step function which is 0 for 0 ≤ x < 1 and 1 for 1 ≤ x ≤ 2, but this step function is discontinuous (Problem 1.8)! ⋄

This shows that in infinite dimensional spaces different norms will give raise to different convergent sequences! In fact, the key to solving problems in infinite dimensional spaces is often finding the right norm! This is something which cannot happen in the finite dimensional case.

Theorem 1.8. If X is a finite dimensional case, then all norms are equivalent. That is, for given two norms ∥.∥_{1} and ∥.∥_{2} there are constants m_{1} and m_{2} such that

\frac{1}{m_{2}} ∥f∥_{1} ≤ ∥f∥_{2} ≤ m_{1}∥f∥_{1}.  (1.36)

Proof. Clearly we can choose a basis u_{j}, 1 ≤ j ≤ n, and assume that ∥.∥_{2} is the usual Euclidean norm, ∥ ∑_{j} α_{j}u_{j}∥_{2}^{2} = ∑_{j} |α_{j}|^{2}. Let f = ∑_{j} α_{j}u_{j}, then by the triangle and Cauchy Schwartz inequalities

∥f∥_{1} ≤ ∑_{j} |α_{j}|∥u_{j}∥_{1} ≤ \sqrt{∑_{j} ∥u_{j}∥_{1}^{2}}∥f∥_{2}  (1.37)

and we can choose m_{2} = \sqrt{∑_{j} ∥u_{j}∥_{1}}.

In particular, if f_{n} is convergent with respect to ∥.∥_{2} it is also convergent with respect to ∥.∥_{1}. Thus ∥.∥_{1} is continuous with respect to ∥.∥_{2} and attains its minimum m > 0 on the unit sphere (which is compact by the Heine-Borel theorem). Now choose m_{1} = 1/m. □

Problem 1.5. Show that ℓ^{2}(ℕ) is a separable Hilbert space.

Problem 1.6. Let s(f, g) be a skew linear form and p(f) = s(f, f) the associated quadratic form. Prove the parallelogram law

p(f + g) + p(f − g) = 2p(f) + 2p(g)  (1.38)

and the polarization identity

s(f, g) = \frac{1}{4} (p(f + g) − p(f − g) + i p(f − ig) − i p(f + ig)) .  (1.39)

Problem 1.7. Show that the maximum norm does not satisfy the parallelogram law.

Problem 1.8. Prove the claims made about f_{n}, defined in (1.35), in the last example.

1.3. Completeness

Since L^{2} is not complete, how can we obtain a Hilbert space out of it? Well the answer is simple: take the completion.

If X is a (incomplete) normed space, consider the set of all Cauchy sequences X̃. Call two Cauchy sequences equivalent if their difference converges to zero and denote by X̄ the set of all equivalence classes. It is easy to see that X̄ (and X̃) inherit the vector space structure from X. Moreover,

Lemma 1.9. If x_{n} is a Cauchy sequence, then ∥x_{n}∥ converges.

Consequently the norm of a Cauchy sequence (x_{n})_{n=1}^{∞} can be defined by ∥(x_{n})_{n=1}^{∞}∥ = lim_{n→∞} ∥x_{n}∥ and is independent of the equivalence class (show this!). Thus X̄ is a normed space (X̃ is not! why?).

Theorem 1.10. X̄ is a Banach space containing X as a dense subspace if we identify x ∈ X with the equivalence class of all sequences converging to x.

Proof. (Outline) It remains to show that X̄ is complete. Let ξ_{n} = [(x_{n,j})_{j=1}^{∞}] be a Cauchy sequence in X̄. Then it is not hard to see that ξ = [(x_{j,j})_{j=1}^{∞}] is its limit. □

In particular it is no restriction to assume that a normed linear space or an inner product space is complete. However, in the important case of L^{2} it is somewhat inconvenient to work with equivalence classes of Cauchy sequences and hence we will give a different characterization using the Lebesgue integral later.

1.4. Bounded operators

A linear map A between two normed spaces X and Y will be called a (linear) operator

A : 𝔇(A) ⊆ X → Y.  (1.40)

The linear subspace 𝔇(A) on which A is defined, is called the domain of A and is usually required to be dense. The operator A is called bounded if the following operator norm

∥A∥ = sup_{∥f∥_{X}=1} ∥Af∥_{Y}  (1.41)

is finite.

The set of all bounded linear operators from X to Y is denoted by 𝔏(X, Y). If X = Y we write 𝔏(X, X) = 𝔏(X).

Theorem 1.11. The space 𝔏(X, Y) together with the operator norm (1.41) is a normed space. It is a Banach space if Y is.

Proof. That (1.41) is indeed a norm is straightforward. If Y is complete and A_{n} is a Cauchy sequence of operators, then A_{n}f converges to an element g for every f. Define a new operator A via Af = g and note A_{n} → A. □

By construction, a bounded operator is Lipschitz continuous

∥Af∥_{Y} ≤ ∥A∥∥f∥_{X}  (1.42)

and hence continuous. The converse is also true

Theorem 1.12. An operator A is bounded if and only if it is continuous.

Proof. Suppose A is continuous but not bounded. Then there is a sequence of unit vectors u_{n} such that ∥Au_{n}∥ ≥ n. Then f_{n} = \frac{1}{n}u_{n} converges to 0 but ∥Af_{n}∥ ≥ 1 does not converge to 0. □

Moreover, if A is bounded and densely defined, it is no restriction to assume that it is defined on all of X.

Theorem 1.13. Let A ∈ 𝔏(X, Y) and let Y be a Banach space. If 𝔇(A) is dense, there is a unique (continuous) extension of A to X, which has the same norm.

Proof. Since a bounded operator maps Cauchy sequences to Cauchy sequences, this extension can only be given by

Af = lim_{n→∞} Af_{n},  f_{n} ∈ 𝔇(A), f ∈ X.  (1.43)

To show that this definition is independent of the sequence f_{n} → f, let g_{n} → f be a second sequence and observe

∥Af_{n} − Ag_{n}∥ = ∥A(f_{n} − g_{n})∥ ≤ ∥A∥∥f_{n} − g_{n}∥ → 0.  (1.44)

From continuity of vector addition and scalar multiplication it follows that our extension is linear. Finally, from continuity of the norm we conclude that the norm does not increase. □

An operator in 𝔏(X, ℂ) is called a bounded linear functional and the space X^{∗} = 𝔏(X, ℂ) is called the dual space of X.

Problem 1.9. Show that the integral operator

(Kf)(x) = ∫_{0}^{1} K(x, y)f(y)dy,  (1.45)

where K(x, y) ∈ C([0, 1] × [0, 1]), defined on 𝔇(K) = C[0, 1] is a bounded operator both in X = C[0, 1] (max norm) and X = L^{2}(0, 1).

Problem 1.10. Show that the differential operator A = \frac{d}{dx} defined on 𝔇(A) = C^{1}[0, 1] ⊂ C[0, 1] is an unbounded operator.

Hilbert spaces

2.1. Orthonormal bases

In this section we will investigate orthonormal series and you will notice hardly any difference between the finite and infinite dimensional cases.

As our first task, let us generalize the projection into the direction of one vector:

A set of vectors {u_{j}} is called orthonormal set if ⟨u_{j}, u_{k}⟩ = 0 for j ≠ k and ⟨u_{j}, u_{j}⟩ = 1.

Lemma 2.1. Suppose {u_{j}}_{j=1}^{n} is an orthonormal set. Then every f ∈ ℌ can be written as

f = f_{∥} + f_{⊥},  f_{∥} = ∑_{j=1}^{n}⟨u_{j}, f⟩u_{j},  (2.1)

where f_{∥} and f_{⊥} are orthogonal. Moreover, ⟨u_{j}, f_{⊥}⟩ = 0 for all 1 ≤ j ≤ n. In particular,

∥f∥^{2} = ∑_{j=1}^{n} |⟨u_{j}, f⟩|^{2} + ∥f_{⊥}∥^{2}.  (2.2)

Moreover, every f̂ in the span of {u_{j}}_{j=1}^{n} satisfies

∥f − f̂∥ ≥ ∥f_{⊥}∥  (2.3)

with equality holding if and only if f̂ = f_{∥}. In other words, f_{∥} is uniquely characterized as the vector in the span of {u_{j}}_{j=1}^{n} being closest to f.

Proof. A straightforward calculation shows ⟨u_{j}, f − f_{∥}⟩ = 0 and hence f_{∥} and f_{⊥} = f − f_{∥} are orthogonal. The formula for the norm follows by applying (1.22) iteratively.

Now, fix a vector

f̂ = ∑_{j=1}^{n} c_{j}u_{j}.  (2.4)

in the span of {u_{j}}_{j=1}^{n}. Then one computes

∥f − f̂∥^{2} = ∥f_{∥} + f_{⊥} − f̂∥^{2} = ∥f_{⊥}∥^{2} + ∥f_{∥} − f̂∥^{2}
= ∥f_{⊥}∥^{2} + ∑_{j=1}^{n} |c_{j} − ⟨u_{j}, f⟩|^{2}  (2.5)

from which the last claim follows. □

From (2.2) we obtain Bessel’s inequality

∑_{j=1}^{n} |⟨u_{j}, f⟩|^{2} ≤ ∥f∥^{2}  (2.6)

with equality holding if and only if f lies in the span of {u_{j}}_{j=1}^{n}.

Of course, since we cannot assume ℌ to be a finite dimensional vector space, we need to generalize Lemma 2.1 to arbitrary orthonormal sets {u_{j}}_{j∈J}. We start by assuming that J is countable. Then Bessel’s inequality (2.6) shows that

∑_{j∈J} |⟨u_{j}, f⟩|^{2}  (2.7)

converges absolutely. Moreover, for any finite subset K ⊂ J we have

∥ ∑_{j∈K}⟨u_{j}, f⟩u_{j}∥^{2} = ∑_{j∈K} |⟨u_{j}, f⟩|^{2}  (2.8)

by the Pythagorean theorem and thus ∑_{j∈J}⟨u_{j}, f⟩u_{j} is Cauchy if and only if ∑_{j∈J} |⟨u_{j}, f⟩|^{2} is. Now let J be arbitrary. Again, Bessel’s inequality shows that for any given ε > 0 there are at most finitely many j for which |⟨u_{j}, f⟩| ≥ ε (namely at most ∥f∥/ε). Hence there are at most countably many j for which |⟨u_{j}, f⟩| > 0. Thus it follows that

∑_{j∈J} |⟨u_{j}, f⟩|^{2}  (2.9)

is well-defined and (by completeness) so is

∑_{j∈J}⟨u_{j}, f⟩u_{j}.  (2.10)

In particular, by continuity of the scalar product we see that Lemma 2.1 holds for arbitrary orthonormal sets without modifications.

Theorem 2.2. Suppose {u_{j}}_{j∈J} is an orthonormal set in an inner product space ℌ. Then every f ∈ ℌ can be written as

f = f_{∥} + f_{⊥},  f_{∥} = ∑_{j∈J}⟨u_{j}, f⟩u_{j},  (2.11)

where f_{∥} and f_{⊥} are orthogonal. Moreover, ⟨u_{j}, f_{⊥}⟩ = 0 for all j ∈ J. In particular,

∥f∥^{2} = ∑_{j∈J} |⟨u_{j}, f⟩|^{2} + ∥f_{⊥}∥^{2}.  (2.12)

Moreover, every f̂ in the span of {u_{j}}_{j∈J} satisfies

∥f − f̂∥ ≥ ∥f_{⊥}∥  (2.13)

with equality holding if and only if f̂ = f_{∥}. In other words, f_{∥} is uniquely characterized as the vector in the span of {u_{j}}_{j∈J} being closest to f.

Note that from Bessel’s inequality (which of course still holds) it follows that the map f → f_{∥} is continuous.

Of course we are particularly interested in the case where every f ∈ ℌ can be written as ∑_{j∈J}⟨u_{j}, f⟩u_{j}. In this case we will call the orthonormal set {u_{j}}_{j∈J} an orthonormal basis.

If ℌ is separable it is easy to construct an orthonormal basis. In fact, if ℌ is separable, then there exists a countable total set {f_{j}}_{j=1}^{N}. After throwing away some vectors we can assume that f_{n+1} cannot be expressed as a linear combinations of the vectors f_{1}, … f_{n}. Now we can construct an orthonormal set as follows: We begin by normalizing f_{1}

u_{1} = \frac{f_{1}}{∥f_{1}∥}.  (2.14)

Next we take f_{2} and remove the component parallel to u_{1} and normalize again

u_{2} = \frac{f_{2} − ⟨u_{1}, f_{2}⟩u_{0}}{∥f_{2} − ⟨u_{1}, f_{2}⟩u_{1}∥}.  (2.15)

Proceeding like this we define recursively

u_{n} = \frac{f_{n} − ∑_{j=1}^{n−1}⟨u_{j}, f_{n}⟩u_{j}}{∥f_{n} − ∑_{j=1}^{n−1}⟨u_{j}, f_{n}⟩u_{j}∥}.  (2.16)

This procedure is known as Gram-Schmidt orthogonalization. Hence we obtain an orthonormal set {u_{j}}_{j=1}^{N} such that span{u_{j}}_{j=1}^{n} = span{f_{j}}_{j=1}^{n} for any finite n and thus also for N. Since {f_{j}}_{j=1}^{N} is total, we infer that {u_{j}}_{j=1}^{N} is an orthonormal basis.

Theorem 2.3. Every separable inner product space has a countable orthonormal basis.

Example. In L^{2}(−1, 1) we can orthogonalize the polynomial f_{n}(x) = x^{n}. The resulting polynomials are up to a normalization equal to the Legendre polynomials

P_{n}(x) = 1, P_{1}(x) = x, P_{2}(x) = \frac{3 x^{2} − 1}{2}, …  (2.17)

(which are normalized such that P_{n}(1) = 1). ⋄

Example. The set of functions

u_{n}(x) = \frac{1}{\sqrt{2π}} e^{in x},  n ∈ ℤ,  (2.18)

forms an orthonormal basis for ℌ = L^{2}(0, 2π). The corresponding orthogonal expansion is just the ordinary Fourier series. ⋄

If fact, if there is one countable basis, then it follows that every other basis is countable as well.

Theorem 2.4. If ℌ is separable, then every orthonormal basis is countable.

Proof. We know that there is at least one countable orthonormal basis {u_{j}}_{j∈J}. Now let {u_{k}}_{k∈K} be a second basis and consider the set K_{j} = {k ∈ K|⟨u_{k}, u_{j}⟩ ≠ 0}. Since these are the expansion coefficients of u_{j} with respect to {u_{k}}_{k∈K}, this set is countable. Hence the set K̃ = ⋃_{j∈J} K_{j} is countable as well. But k ∈ K\K̃ implies u_{k} = 0 and hence K̃ = K. □

It even turns out that, up to unitary equivalence, there is only one (separable) infinite dimensional Hilbert space:

A bijective operator U ∈ 𝔏(ℌ_{1}, ℌ_{2}) is called unitary if U preserves scalar products:

⟨Ug, Uf⟩_{2} = ⟨g, f⟩_{1},  g, f ∈ ℌ_{1}.  (2.19)

By the polarization identity this is the case if and only if U preserves norms: ∥Uf∥_{2} = ∥f∥_{1} for all f ∈ ℌ_{1}. The two Hilbert space ℌ_{1} and ℌ_{2} are called unitarily equivalent in this case.

Let ℌ be an infinite dimensional Hilbert space and let {u_{j}}_{j∈ℕ} be any orthogonal basis. Then the map U : ℌ → ℓ^{2}(ℕ), f ↦ (⟨u_{j}, f⟩)_{j∈ℕ} is unitary (by Theorem 2.6 (iii)). In particular,

Theorem 2.5. Any separable infinite dimensional Hilbert space is unitarily equivalent to ℓ^{2}(ℕ).

To see that any Hilbert space has an orthonormal basis we need to resort to Zorn’s lemma: The collection of all orthonormal sets in ℌ can be partially ordered by inclusion. Moreover, any linearly ordered chain has an upper bound (the union of all sets in the chain). Hence Zorn’s lemma implies the existence of a maximal element, that is, an orthonormal set which is not a proper subset of any other orthonormal set.

Theorem 2.6. For an orthonormal set {u_{j}}_{j∈J} in an Hilbert space ℌ the following conditions are equivalent:

(i) {u_{j}}_{j∈J} is a maximal orthogonal set. (ii) For every vector f ∈ ℌ we have

f = ∑_{j∈J}⟨u_{j}, f⟩u_{j}.  (2.20)

(iii) For every vector f ∈ ℌ we have

∥f∥^{2} = ∑_{j∈J} |⟨u_{j}, f⟩|^{2}.  (2.21)

(iv) ⟨u_{j}, f⟩ = 0 for all j ∈ J implies f = 0.

Proof. We will use the notation from Theorem 2.2.

(i) ⇒ (ii): If f_{⊥} ≠ 0 than we can normalize f_{⊥} to obtain a unit vector f̃_{⊥} which is orthogonal to all vectors u_{j}. But then {u_{j}}_{j∈J} ∪ {f̃_{⊥}} would be a larger orthonormal set, contradicting maximality of {u_{j}}_{j∈J}. (ii) ⇒ (iii): Follows since (ii) implies f_{⊥} = 0. (iii) ⇒ (iv): If ⟨f, u_{j}⟩ = 0 for all j ∈ J we conclude ∥f∥^{2} = 0 and hence f = 0. (iv) ⇒ (i): If {u_{j}}_{j∈J} were not maximal, there would be a unit vector g such that {u_{j}}_{j∈J} ∪ {g} is larger orthonormal set. But ⟨u_{j}, g⟩ = 0 for all j ∈ J implies g = 0 by (iv), a contradiction. □

By continuity of the norm it suffices to check (iii), and hence also (ii), for f in a dense set.

2.2. The projection theorem and the Riesz lemma

Let M ⊆ ℌ be a subset, then M^{⊥} = {f|⟨g, f⟩ = 0, ∀g ∈ M} is called the orthogonal complement of M. By continuity of the scalar product it follows that M^{⊥} is a closed linear subspace and by linearity that (\overline{span(M)})^{⊥} = M^{⊥}. For example we have ℌ^{⊥} = \{0\} since any vector in ℌ^{⊥} must be in particular orthogonal to all vectors in some orthonormal basis.

Theorem 2.7 (projection theorem). Let M be a closed linear subspace of a Hilbert space ℌ, then every f ∈ ℌ can be uniquely written as f = f_{∥} + f_{⊥} with f_{∥} ∈ M and f_{⊥} ∈ M^{⊥}. One writes

M ⊕ M^{⊥} = ℌ  (2.22)

in this situation.

Proof. Since M is closed, it is a Hilbert space and has an orthonormal basis {u_{j}}_{j∈J}. Hence the result follows from Theorem 2.2. □

In other words, to every f ∈ ℌ we can assign a unique vector f_{∥} which is the vector in M closest to f. The rest f − f_{∥} lies in M^{⊥}. The operator P_{M}f = f_{∥} is called the orthogonal projection corresponding to M. Clearly we have P_{M^{⊥}}f = f − P_{M}f = f_{⊥}.

Moreover, we see that the vectors in a closed subspace M are precisely those which are orthogonal to all vectors in M^{⊥}, that is, M^{⊥⊥} = M. If M is an arbitrary subset we have at least

M^{⊥⊥} = \overline{span(M)}.  (2.23)

Finally we turn to linear functionals, that is, to operators ℓ : ℌ → ℂ. By the Cauchy-Schwarz inequality we know that ℓ_{g} : f ↦ ⟨g, f⟩ is a bounded linear functional (with norm ∥g∥). In turns out that in a Hilbert space every bounded linear functional can be written in this way.

Theorem 2.8 (Riesz lemma). Suppose ℓ is a bounded linear functional on a Hilbert space ℌ. Then there is a vector g ∈ ℌ such that ℓ(f) = ⟨g, f⟩ for all f ∈ ℌ. In other words, a Hilbert space is equivalent to its own dual space ℌ^{∗} = ℌ.

Proof. If ℓ ≡ 0 we can choose g = 0. Otherwise Ker(ℓ) = {f|ℓ(f) = 0} is a proper subspace and we can find a unit vector g̃ ∈ Ker(ℓ)^{⊥}. For every f ∈ ℌ we have ℓ(f)g̃ − ℓ(g̃)f ∈ Ker(ℓ) and hence

0 = ⟨g̃, ℓ(f)g̃ − ℓ(g̃)f⟩ = ℓ(f) − ℓ(g̃)⟨g̃, f⟩.  (2.24)

In other words, we can choose g = ℓ(g̃)^{∗}g̃. □

The following easy consequence is left as an exercise.

Corollary 2.9. Suppose B is a bounded skew liner form, that is,

|B(f, g)| ≤ C∥f∥ ∥g∥.  (2.25)

Then there is a unique bounded operator A such that

B(f, g) = ⟨Af, g⟩.  (2.26)

2.3. Orthogonal sums and tensor products

Given two Hilbert spaces ℌ_{1} and ℌ_{2} we define their orthogonal sum ℌ_{1}⊕ℌ_{2} to be the set of all pairs (f_{1}, f_{2}) ∈ ℌ_{1} × ℌ_{2} together with the scalar product

⟨(g_{1}, g_{2}), (f_{1}, f_{2})⟩ = ⟨g_{1}, f_{1}⟩_{1} + ⟨g_{2}, f_{2}⟩_{2}.  (2.27)

It is left as an exercise to verify that ℌ_{1} ⊕ ℌ_{2} is again a Hilbert space. Moreover, ℌ_{1} can be identified with {(f_{1}, 0)|f_{1} ∈ ℌ_{1}} and we can regard ℌ_{1} as a subspace of ℌ_{1} ⊕ ℌ_{2}. Similarly for ℌ_{2}. It is also custom to write f_{1} + f_{2} instead of (f_{1}, f_{2}).

More generally, let ℌ_{j} j ∈ ℕ, be a countable collection of Hilbert spaces and define

⨁_{j=1}^{∞} ℌ_{j} = {∑_{j=1}^{∞} f_{j}| f_{j} ∈ ℌ_{j}, ∑_{j=1}^{∞} ∥f_{j}∥_{j}^{2} < ∞},  (2.28)

which becomes a Hilbert space with the scalar product

⟨∑_{j=1}^{∞} g_{j}, ∑_{j=1}^{∞} f_{j}⟩ = ∑_{j=1}^{∞}⟨g_{j}, f_{j}⟩_{j}.  (2.29)

Example. ⨁_{j=1}^{∞} ℂ = ℓ^{2}(ℕ). ⋄

Suppose ℌ and ℌ̃ are two Hilbert spaces. Our goal is to construct their tensor product. The elements should be products f ⊗ f̃ of elements f ∈ ℌ and f̃ ∈ ℌ̃. Hence we start with the set of all finite linear combinations of elements of ℌ × ℌ̃

F(ℌ, ℌ̃) = {∑_{j=1}^{n} α_{j}(f_{j}, f̃_{j})|(f_{j}, f̃_{j}) ∈ ℌ × ℌ̃, α_{j} ∈ ℂ}.  (2.30)

Since we want (f_{1} + f_{2}) ⊗ f̃ = f_{1} ⊗ f̃+ f_{2} ⊗ f̃, f ⊗ (f̃_{1} + f̃_{2}) = f ⊗ f̃_{1} + f ⊗ f̃_{2}, and (αf) ⊗ f̃ = f ⊗ (αf̃) we consider F(ℌ, ℌ̃)/N (ℌ, ℌ̃), where

N (ℌ, ℌ̃) = span{ ∑_{j,k=1}^{n} α_{j}β_{k}(f_{j}, f̃_{k}) − (∑_{j=1}^{n} α_{j}f_{j}, ∑_{k=1}^{n} β_{k}f̃_{k})}  (2.31)

and write f ⊗ f̃ for the equivalence class of (f, f̃).

Next we define

⟨f ⊗ f̃, g ⊗ g̃⟩ = ⟨f, g⟩⟨f̃, g̃⟩  (2.32)

which extends to a skew linear form on F(ℌ, ℌ̃)/N (ℌ, ℌ̃). To show that we obtain a scalar product, we need to ensure positivity. Let f = ∑_{i} α_{i}f_{i}⊗f̃_{i} ≠ 0 and pick orthonormal bases u_{j}, ũ_{k} for span{f_{i}}, span{f̃_{i}}, respectively. Then

f = ∑_{j,k} α_{jk}u_{j} ⊗ ũ_{k}, α_{jk} = ∑_{i} α_{i}⟨u_{j}, f_{i}⟩⟨ũ_{k}, f̃_{i}⟩  (2.33)

and we compute

⟨f, f⟩ = ∑_{j,k} |α_{jk}|^{2} > 0.  (2.34)

The completion of F(ℌ, ℌ̃)/N (ℌ, ℌ̃) with respect to the induced norm is called the tensor product ℌ ⊗ ℌ̃ of ℌ and ℌ̃.

Lemma 2.10. If u_{j}, ũ_{k} are orthonormal bases for ℌ, ℌ̃, respectively, then u_{j} ⊗ ũ_{k} is an orthonormal basis for ℌ ⊗ ℌ̃.

Proof. That u_{j} ⊗ ũ_{k} is an orthonormal set is immediate from (2.32). Moreover, since span{u_{j}}, span{ũ_{k}} is dense in ℌ, ℌ̃, respectively, it is easy to see that u_{j} ⊗ ũ_{k} is dense in F(ℌ, ℌ̃)/N (ℌ, ℌ̃). But the latter is dense in ℌ ⊗ ℌ̃. □

Example. We have ℌ ⊗ ℂ^{n} = ℌ^{n}. ⋄

It is straightforward to extend the tensor product to any finite number of Hilbert spaces. We even note

(⨁_{j=1}^{∞} ℌ_{j}) ⊗ ℌ = ⨁_{j=1}^{∞}(ℌ_{j} ⊗ ℌ),  (2.35)

where equality has to be understood in the sense, that both spaces are unitarily equivalent by virtue of the identification

(∑_{j=1}^{∞} f_{j}) ⊗ f = ∑_{j=1}^{∞} f_{j} ⊗ f.  (2.36)

2.4. Compact operators

A linear operator A defined on a normed space X is called compact if every sequence Af_{n} has a convergent subsequence whenever f_{n} is bounded. The set of all compact operators is denoted by ℭ(X). It is not hard to see that the set of compact operators is a ideal of the set of bounded operators (Problem 2.1):

Theorem 2.11. Every compact linear operator is bounded. Linear combinations of compact operators are bounded and the product of a bounded and a compact operator is again compact.

If X is a Banach space then this ideal is even closed:

Theorem 2.12. Let X be a Banach space, and let A_{n} be a convergent sequence of compact operators. Then the limit A is again compact.

Proof. Let f_{j}^{(0)} be a bounded sequence. Chose a subsequence f_{j}^{(1)} such

that A_{1}f_{j}^{(1)} converges. From f_{j}^{(1)} choose another subsequence f_{j}^{(1)} such that

A_{2}f_{j}^{(2)} converges and so on. Since f_{j}^{(n)} might disappear as n → ∞, we con-

sider the diagonal sequence f_{j} = f_{j}^{(j)}. By construction, f_{j} is a subsequence

of f_{j}^{(n)} for j ≥ n and hence A_{n}f_{j} is Cauchy for any fixed n. Now

∥Af_{j} − f_{k}∥ = ∥(A − A_{n})(f_{j} − f_{k}) + A_{n}(f_{j} − f_{k})∥
≤ ∥A − A_{n}∥∥f_{j} − f_{k}∥ + ∥A_{n}f_{j} − A_{n}f_{k}∥  (2.37)

shows that Af_{j} is Cauchy since the first term can be made arbitrary small by choosing n large and the second by the Cauchy property of A_{n}f_{j}. □

Note that it suffices to verify compactness on a dense set.

Theorem 2.13. Let X be a normed space and A ∈ ℭ(X). Let \overline{X} be its completion, then \overline{A} ∈ ℭ(\overline{X}), where \overline{A} is the unique extension of A.

Proof. Let f_{n} ∈ \overline{X} be a given bounded sequence. We need to show that \overline{A}f_{n} has a convergent subsequence. Pick f_{n}^{j} ∈ X such that ∥f_{n}^{j}−f_{n}∥ ≤ \frac{1}{j} and by compactness of A we can assume that Af_{n}^{n} → g. But then ∥\overline{A}f_{n} − g∥ ≤ ∥A∥∥f_{n} − f_{n}^{n}∥ + ∥Af_{n}^{n} − g∥ shows that \overline{A}f_{n} → g. □

One of the most important examples of compact operators are integral operators:

Lemma 2.14. The integral operator

(Kf)(x) = ∫_{a}^{b} K(x, y)f(y)dy,  (2.38)

where K(x, y) ∈ C([a, b] × [a, b]), defined on L^{2}(a, b) is compact.

Proof. First of all note that K(., ..) is continuous on [a, b] × [a, b] and hence uniformly continuous. In particular, for every ε > 0 we can find a δ > 0 such that |K(y, t) − K(x, t)| ≤ ε whenever |y − x| ≤ δ. Let g(x) = Kf(x), then

|g(x) − g(y)| ≤ ∫_{a}^{b} |K(y, t) − K(x, t)| |f(t)|dt
≤ ε ∫_{a}^{b} |f(t)|dt ≤ ε∥1∥ ∥f∥,  (2.39)

whenever |y − x| ≤ δ. Hence, if f_{n}(x) is a bounded sequence in L^{2}(a, b), then g_{n}(x) = Kf_{n}(x) is equicontinuous and has a uniformly convergent subsequence by the Arzelà-Ascoli theorem (Theorem 2.15 below). But a uniformly convergent sequence is also convergent in the norm induced by the scalar product. Therefore K is compact. □

Note that (almost) the same proof shows that K is compact when defined on C[a, b].

Theorem 2.15 (Arzelà-Ascoli). Suppose the sequence of functions f_{n}(x), n ∈ ℕ, on a compact interval is (uniformly) equicontinuous, that is, for every ε > 0 there is a δ > 0 (independent of n) such that

|f_{n}(x) − f_{n}(y)| ≤ ε if |x − y| < δ.  (2.40)

If the sequence f_{n} is bounded, then there is a uniformly convergent subsequence.

(The proof is not difficult but I still don’t want to repeat it here since it is covered in most real analysis courses.)

Compact operators are very similar to (finite) matrices as we will see in the next section.

Problem 2.1. Show that compact operators form an ideal.

2.5. The spectral theorem for compact symmetric operators

Let ℌ be an inner product space. A linear operator A is called symmetric if its domain is dense and if

⟨g, Af⟩ = ⟨Ag, f⟩  f, g ∈ 𝔇(A).  (2.41)

A number z ∈ ℂ is called eigenvalue of A if there is a nonzero vector u ∈ 𝔇(A) such that

Au = zu.  (2.42)

The vector u is called a corresponding eigenvector in this case. An eigenvalue is called simple if there is only one linearly independent eigenvector.

Theorem 2.16. Let A be symmetric. Then all eigenvalues are real and eigenvectors corresponding to different eigenvalues are orthogonal.

Proof. Suppose λ is an eigenvalue with corresponding normalized eigenvector u. Then λ = ⟨u, Au⟩ = ⟨Au, u⟩ = λ^{∗}, which shows that λ is real. Furthermore, if Au_{j} = λ_{j}u_{j}, j = 1, 2, we have

(λ_{1} − λ_{2})⟨u_{1}, u_{2}⟩ = ⟨Au_{1}, u_{2}⟩ − ⟨u_{1}, Au_{2}⟩ = 0  (2.43)

finishing the proof. □

Now we show that A has an eigenvalue at all (which is not clear in the infinite dimensional case)!

Theorem 2.17. A symmetric compact operator has an eigenvalue α_{1} which satisfies |α_{1}| = ∥A∥.

Proof. We set α = ∥A∥ and assume α ≠ 0 (i.e, A ≠ 0) without loss of generality. Since

∥A∥^{2} = sup_{f:∥f∥=1} ∥Af∥^{2} = sup_{f:∥f∥=1}⟨Af, Af⟩ = sup_{f:∥f∥=1}⟨f, A^{2}f⟩  (2.44)

there exists a normalized sequence u_{n} such that

lim_{n→∞}⟨u_{n}, A^{2}u_{n}⟩ = α^{2}.  (2.45)

Since A is compact, it is no restriction to assume that A^{2}u_{n} converges, say lim_{n→∞} A^{2}u_{n} = α^{2}u. Now

∥(A^{2} − α^{2})u_{n}∥^{2} = ∥A^{2}u_{n}∥^{2} − 2α^{2}⟨u_{n}, A^{2}u_{n}⟩ + α^{4}
≤ 2α^{2}(α^{2} − ⟨u_{n}, A^{2}u_{n}⟩)  (2.46)

(where we have used ∥A^{2}u_{n}∥ ≤ ∥A∥∥Au_{n}∥ ≤ ∥A∥^{2}∥u_{n}∥ = α^{2}) implies lim_{n→∞}(A^{2}u_{n} − α^{2}u_{n}) = 0 and hence lim_{n→∞} u_{n} = u. In addition, u is a normalized eigenvector of A^{2} since (A^{2} − α^{2})u = 0. Factorizing this last equation according to (A − α)u = v and (A + α)v = 0 show that either v ≠ 0 is an eigenvector corresponding to −α or v = 0 and hence u ≠ 0 is an eigenvector corresponding to α. □

Note that for a bounded operator A, there cannot be an eigenvalue with absolute value larger than ∥A∥, that is, the set of eigenvalues is bounded by ∥A∥ (Problem 2.2).

Now consider a symmetric compact operator A with eigenvalue α_{1} (as above) and corresponding normalized eigenvector u_{1}. Setting

ℌ_{1} = {u_{1}}^{⊥} = {f ∈ ℌ|⟨u_{1}, f⟩ = 0}  (2.47)

we can restrict A to ℌ_{1} using

𝔇(A_{1}) = 𝔇(A) ∩ ℌ_{1} = {f ∈ 𝔇(A)|⟨u_{1}, f⟩ = 0}  (2.48)

since f ∈ 𝔇(A_{1}) implies

⟨u_{1}, Af⟩ = ⟨Au_{1}, f⟩ = α_{1}⟨u_{1}, f⟩ = 0  (2.49)

and hence Af ∈ ℌ_{1}. Denoting this restriction by A_{1}, it is not hard to see that A_{1} is again a symmetric compact operator. Hence we can apply Theorem 2.17 iteratively to obtain a sequence of eigenvalues α_{j} with corresponding normalized eigenvectors u_{j}. Moreover, by construction, u_{n} is orthogonal to all u_{j} with j < n and hence the eigenvectors {u_{j}} form an orthonormal set. This procedure will not stop unless ℌ is finite dimensional. However, note that α_{j} = 0 for j ≥ n might happen if A_{n} = 0.

Theorem 2.18. Suppose ℌ is a Hilbert space and A : ℌ → ℌ is a compact symmetric operator. Then there exists a sequence of real eigenvalues α_{j} converging to 0. The corresponding normalized eigenvectors u_{j} form an orthonormal set and every f ∈ ℌ can be written as

f = ∑_{j=1}^{∞}⟨u_{j}, f⟩u_{j} + h,  (2.50)

where h is in the kernel of A, that is, Ah = 0.

In particular, if 0 is not an eigenvalue, then the eigenvectors form an orthonormal basis (in addition, ℌ needs not to be complete in this case).

Proof. Existence of the eigenvalues α_{j} and the corresponding eigenvectors has already been established. If the eigenvalues should not converge to zero, there is a subsequence such that v_{k} = α_{j_{k}}^{−1}u_{j_{k}} is a bounded sequence for which Av_{k} has no convergent subsequence since ∥Av_{k} − Av_{l}∥^{2} = ∥u_{j_{k}} − u_{j_{l}}∥^{2} = 2.

Next, setting

f_{n} = ∑_{j=1}^{n}⟨u_{j}, f⟩u_{j},  (2.51)

we have

∥A(f − f_{n})∥ ≤ |α_{n}|∥f − f_{n}∥ ≤ |α_{n}|∥f∥  (2.52)

since f − f_{n} ∈ ℌ_{n}. Letting n → ∞ shows A(f_{∞} − f) = 0 proving (2.50). □

Remark: There are two cases where our procedure might fail to construct an orthonormal basis of eigenvectors. One case is where there is an infinite number of nonzero eigenvalues. In this case α_{n} never reaches 0 and all eigenvectors corresponding to 0 are missed. In the other case, 0 is reached, but there might not be a countable basis and hence again some of the eigenvectors corresponding to 0 are missed. In any case one can show that by adding vectors from the kernel (which are automatically eigenvectors), one can always extend the eigenvectors u_{j} to an orthonormal basis of eigenvectors.

This is all we need and it remains to apply these results to Sturm-Liouville operators.

Problem 2.2. Show that if A is bounded, then every eigenvalue α satisfies |α| ≤ ∥A∥.

2.6. Applications to Sturm-Liouville operators

Now, after all this hard work, we can show that our Sturm-Liouville operator

L = −\frac{d^{2}}{dx^{2}} + q(x),  (2.53)

where q is continuous and real, defined on

𝔇(L) = {f ∈ C^{2}[0, 1]|f(0) = f(1) = 0} ⊂ L^{2}(0, 1)  (2.54)

has an orthonormal basis of eigenfunctions.

We first verify that L is symmetric:

⟨f, Lg⟩ = ∫_{0}^{1} f(x)^{∗}(−g′′(x) + q(x)g(x))dx
= ∫_{0}^{1} f′(x)^{∗}g′(x)dx + ∫_{0}^{1} f(x)^{∗}q(x)g(x)dx
= ∫_{0}^{1} −f′′(x)^{∗}g(x)dx + ∫_{0}^{1} f(x)^{∗}q(x)g(x)dx  (2.55)
= ⟨Lf, g⟩.

Here we have used integration by part twice (the boundary terms vanish due to our boundary conditions f(0) = f(1) = 0 and g(0) = g(1) = 0).

Next we would need to show that L is bounded. But this task is bound to fail, since L is not even bounded (Problem 1.10)!

So here comes the trick: If L is bounded its inverse L^{−1} might still be bounded. L^{−1} might even be compact an this is the case here! Since L might not be injective (0 might be an eigenvalue), we consider R_{L}(z) = (L − z)^{−1}, z ∈ ℂ, which is also known as the resolvent of L.

A straightforward computation

f(x) = \frac{u_{+}(z, x)}{W (z)}( ∫_{0}^{x} u_{−}(z, t)g(t)dt)
+ \frac{u_{−}(z, x)}{W (z)}( ∫_{x}^{1} u_{+}(z, t)g(t)dt),  (2.56)

verifies that f satisfies (L − z)f = g, where u_{±}(z, x) are the solutions of the homogenous differential equation −u′′_{±}(z, x)+(q(x)−z)u_{±}(z, x) = 0 satisfying the initial conditions u_{−}(z, 0) = 0, u′_{−}(z, 0) = 1 respectively u_{+}(z, 1) = 0, u′_{+}(z, 1) = 1 and

W (z) = W (u_{+}(z), u_{−}(z)) = u′_{−}(z, x)u_{+}(z, x) − u_{−}(z, x)u′_{+}(z, x) (2.57)

is the Wronski determinant, which is independent of x (check this!).

Note that z is an eigenvalue if and only if W (z) = 0. In fact, in this case u_{+}(z, x) and u_{−}(z, x) are linearly dependent and hence u_{−}(z, 0) = cu_{+}(z, 0) = 0 which shows that u_{−}(z) satisfies both boundary conditions and is thus an eigenfunction.

Introducing the the Green function

G(z, x, t) = \frac{1}{W (u_{+}(z), u_{−}(z))} \{ \begin{matrix}u_{+}(z, x)u_{−}(z, t), & x ≥ t \\ u_{+}(z, t)u_{−}(z, x), & x ≤ t \end{matrix}  (2.58)

we see that (L − z)^{−1} is given by

(L − z)^{−1}g(x) = ∫_{0}^{1} G(z, x, t)g(t)dt.  (2.59)

Moreover, from G(z, x, t) = G(z, t, x) it follows that (L − z)^{−1} is symmetric for z ∈ ℝ (Problem 2.3) and from Lemma 2.14 it follows that K is compact. Hence Theorem 2.18 applies to (L − z)^{−1} and we obtain:

Theorem 2.19. The Sturm-Liouville operator L has a countable number of eigenvalues E_{n}. All eigenvalues are discrete and simple. The corresponding normalized eigenfunctions u_{n} form an orthonormal basis for L^{2}(0, 1).

Proof. Pick a value λ ∈ ℝ such that R_{L}(λ) exists. By Theorem 2.18 there are eigenvalues α_{n} of R_{L}(λ) with corresponding eigenfunctions u_{n}. Moreover, R_{L}(λ)u_{n} = α_{n}u_{n} is equivalent to Lu_{n} = (λ + \frac{1}{α_{n}})u_{n}, which shows that E_{n} = λ + \frac{1}{α_{n}} are eigenvalues of L with corresponding eigenfunctions u_{n}. Now everything follows from Theorem 2.18 except that the eigenvalues are simple. To show this, observe that if u_{n} and v_{n} are two different eigenfunctions corresponding to E_{n}, then u_{n}(0) = v_{n}(0) = 0 implies W (u_{n}, v_{n}) = 0 and hence u_{n} and v_{n} are linearly dependent. □

Problem 2.3. Show that the integral operator

(Kf)(x) = ∫_{0}^{1} K(x, y)f(y)dy,  (2.60)

where K(x, y) ∈ C([0, 1] × [0, 1]) is symmetric if K(x, y)^{∗} = K(x, y).

Chapter 3

Banach spaces Bibliography

[1] H. Heuser, Funktionalanalysis, B.G.Teubner, Stuttgart, 1986.

[2] M. Reed and B. Simon, Methods of Modern Mathematical Physics I. Functional Analysis, rev. and enl. edition, Academic Press, San Diego, 1980.

[3] W. Rudin, Real and Complex Analysis, 3rd edition, McGraw-Hill, New York, 1987.

[4] J. Weidmann, Lineare Operatoren in Hilberträumen I: Grundlagen, B.G.Teubner, Stuttgart, 2000.

[5] D. Werner, Funktionalanalysis, 3rd edition., Springer, Berlin, 2000.

Glossary of notations

ℭ(ℌ)  … set of compact operators, 24.
C(U)  … set of continuous functions from U to ℂ.
C(U, V) … set of continuous functions from U to V.
C_{0}^{∞}(U, V) … set of compactly supported smooth functions
χ_{Ω}(.)  … characteristic function of the set Ω
dim  … dimension of a linear space
𝔇(.)  … domain of an operator
e  … exponential function, e^{z} = exp(z)
hull(.)  … convex hull
ℌ  … a Hilbert space
i  … complex unity, i^{2} = −1
Im(.)  … imaginary part of a complex number
inf  … infimum
Ker(A)  … kernel of an operator A
𝔏(X, Y) … set of all bounded linear operators from X to Y, 14
𝔏(X)  = 𝔏(X, X)
max  … maximum
ℕ  … the set of positive integers
ℕ_{0}  = ℕ ∪ {0}
Ran(A) … range of an operator A
Re(.)  … real part of a complex number
ℝ  … the set of real numbers
sup  … supremum
supp  … support of a function
span(M) … set of finite linear combinations from M
ℤ  … the set of integers

𝕀  … identity operator
\sqrt{z}  … square root of z with branch cut along (−∞, 0)
z^{∗}  … complex conjugation
∥.∥  … norm
∥.∥_{p}  … norm in the Banach space L^{p}
⟨., ..⟩  … scalar product in ℌ
⊕  … orthogonal sum of linear spaces or operators, 23
∂  … gradient
∂_{α}  … partial derivative
M^{⊥}  … orthogonal complement, 21
(λ_{1}, λ_{2}) = {λ ∈ ℝ | λ_{1} < λ < λ_{2}}, open interval
[λ_{1}, λ_{2}] = {λ ∈ ℝ | λ_{1} ≤ λ ≤ λ_{2}}, closed interval

Index

Banach space, 6 Basis

orthonormal, 19 Bessel inequality, 18 Boundary condition, 4 Boundary value problem, 4

Cauchy-Schwarz inequality, 11 Complete, 6

Diffusion equation, 1 Domain, 14

Eigenvalue, 26

simple, 26 Eigenvector, 26

Fourier series, 20

Gram-Schmidt orthogonalization, 19 Green function, 29

Heat equation, 1 Hilbert space, 9

Inner product, 9 Inner product space, 9

Linear functional, 15, 22

Norm, 5

operator, 14 Normalized, 10 Normed space, 5

Operator bounded, 14 compact, 24 domain, 14 linear, 14 symmetric, 26 unitary, 20 Orthogonal, 10 Orthogonal complement, 21 Orthogonal sum, 23

Parallel, 10

Parallelogram law, 11 Perpendicular, 10 Polarization identity, 11 Pythagorean theorem, 10

Resolvent, 29 Riesz lemma, 22

Scalar product, 9 Separable, 7 Separation of variables, 2 Sturm–Liouville problem, 4

Tensor product, 24

Theorem

Arzelà-Ascoli, 26 Total, 7 Triangel inequality, 5

Unit vector, 10

Wave equation, 3

Weierstraß approxiamation, 7